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ABSTRACT 

The current aircraft assessment process typically 
makes extensive use of operational personnel during 
simulations and operational evaluations, with 
increased emphasis on evaluating the many pilot 
and/or operator/aircraft control loops. The need for a 
crew assessment in this broader arena has produced a 
variety of rating scales. The Cooper-Harper Rating 
Scale is frequently misused and routinely overlooked 
in the process, for these applications often extend the 
scale's use beyond its originally intended application. 
This paper agrees with the broader application of the 
Cooper-Harper Rating Scale and presents a concept for 
the development of a "use unique" Interpreted Cooper- 
Harper Scale to help achieve this objective. This 
interpreted scale concept was conceived during efforts 
to support an FAA evaluation of a night vision 
enhancement system. It includes descriptive 
extensions, which are faithful to the intent of the 
current Cooper-Harper Scale and should provide the 
kind of detail that has historically been provided by 
trained test pilots in their explanatory comments. 


NOMENCLATURE 


CHPRS 

Cooper-Harper Pilot Rating Scale 

CM 

Cockpit Management 

DI 

Deck Interface 

EMS 

Emergency Medical Service 

FAA 

Federal Aviation Administration 

HQR 

Handling Quality Rating 

ICHRS 

Interpreted Cooper-Harper Rating Scale 

IFR 

Instrument Flight Rules 

IMC 

Instrument Meteorological Conditions 

IPR 

Interpreted Pilot Rating 

LFE 

Limit Flight Envelope 

MCHRS 

Modified Cooper-Harper Rating Scale 

NAS 

National Airspace System 

NFE 

Normal Flight Envelope 

NOE 

Nap-of-the-Earth 

OFE 

Operational Right Envelope 

PR 

Pilot Rating 

VMC 

Visual Meteorological Conditions 
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INTRODUCTION 

The Cooper-Harper Pilot Rating Scale (CHPRS) has 
been very effective in handling qualities research and 
development applications, serving as an evaluation 
tool and communications medium in a community of 
trained experimental and R&D test pilots and 
engineers. The success of CHPRS has in some 
measure been due to the discipline involved in its use. 
This discipline has been instilled thorough training at 
test pilot schools, use of the scale in the military 
acquisition process, and because of adherence to the 
asterisk note which appears on the scale: "Definition 
of required operation involves designation of flight 
phase and sub-phases with accompanying conditions". 

In Reference 1, Harper and Cooper emphasize the 
need to follow this stricture. While recognizing the 
difficulties in doing so, they also recognize the adverse 
impact of failure to treat this instruction in a 
comprehensive way. 

Currently, the assessment process in new product 
development for aircraft has taken on a greater 
operational flavor. This is found both at the project 
initiation stage, where extensive simulations using 
operational personnel are becoming the rule, and at the 
final approval stage where operational personnel hold 
the final stamp. At the same time, the greatly 
increased integrated complexity of the pilot machine 
interface systems increases the emphasis on evaluating 
the many in-flight dynamic components of the pilot 
and/or operator/aircraft control loop. This complexity 
is amplified for rotorcraft, where the total flight regime 
includes the widest variety of flight path tasks. 

The need for a rating scale in this broader arena has 
required the use of evaluation scales of some sort, and 
perhaps because piloting considerations are generally 
involved — but not always — the Cooper-Harper scale 
is frequently used. Sometimes it is misused in this 
broader context. Sometimes it is not applied because 
of concern for misuse, or a bureaucratic constraint or 
because it is simply not understood. 

To those who have been trained in the use of the scale, 
it is clear and provides a concise and useful way for 
members of the handling qualities community to 
communicate. To many outside the handling qualities 
community, a reluctance to apply the scale is evoked 
by a lack of confidence in the use of subjective pilot 
evaluations. This group typically desires to use a pass 
fail criteria pilot (crew) evaluation or alternately base 


221 


decisions on quantitative measures alone. It appears 
that the reservations of some are reinforced by their 
unsuccessful attempts to use the scale. These attempts 
may have failed to observe the asterisked stricture of 
the CHPRS (see Figure 1). 

A case can be made for using some other scale, or 
using the CHPRS with a second overlapping workload 
scale, or using no subjective scale at all. But because 
handling qualities are major components of all aircraft 
pilot/operator assessments, and because the scale has 
always included consideration of workload, it seems 
most appropriate to improve our understanding of the 
existing CHPRS and broaden its applications. To this 
end, this paper proposes that a well understood, 
expanded and interpreted version of the CHPRS 
would: 

(1) Help the aviation community define the factors 
which respond to the asterisk note on the CHPRS, 
minimizing variance in pilot ratings. 

(2) Include a concept which involves developing 
"application unique" extensions to the descriptive 
content of the scale to enhance its use by both 
trained engineering test pilots and by operational 
evaluation pilots. These expanded definitions will 
allow pilots to: 

(a) Select a correct rating which may be a whole 
number or a half pilot rating (PR), and 

(b) provide additional comments which will help 
others understand the experience underlying 
the selected rating (in terms which include 
flying qualities, flying workload, cockpit 
management (CM) workload and relevant 
performance measures). 

(3) Better explain how experienced subject pilots can 
predict the suitability of an aircraft for operations 
in environments not specifically evaluated. 

In summary, paper supports the broader application of 
the current CHPRS and it offers a concept for 
achieving this objective through the introduction of an 
use-specific. Interpreted Cooper-Harper Pilot Rating 
Scale. 

COOPER-HARPER RATINGS 
Background 

The first widely used pilot rating scale was introduced 
in 1957 and known as the Cooper Scale (Reference 2). 
This was followed by an interim scale in 1966 
(Reference 3) and finally in 1969 the Cooper-Harper 
rating scale, presented here as Figure 1, was published 
in NASA TN D-5153 (Reference 4). 

The key to effective use of this scale lies in strict 
adherence to the guidelines contained in References 1 
and 4, and in the thorough understanding of the scale’s 
origins, strengths and limitations. In this regard, 
Harper and Cooper reported in Reference 1 that the 
"nearly universal use of the Cooper-Harper rating scale 
for handling qualities assessments is not commen- 
surate with the general lack of access to and familiarity 


with NASA TN D-5153 (which gives background 
guidance, definition of terms, and recommended use)". 
In other words, everybody uses the scale, but few have 
studied Reference 4 and/or observe the counsel of 
Reference 4. 

It is important to understand that most of the ideas and 
suggestions in this paper are not new. For the most 
part, they are over 30 years old and alluded to in the 
above references. This paper does provide suggested 
ways to implement the guidance of References 1 and 4 
as well as expanding the application of the scale to 
address the current needs of the industry. In this 
regard, the following paragraphs quote, paraphrase, 
and amplify a number of key concepts and instructions 
contained in the primary references: 

A Communication Enhancement Tool 

There are two parts to the rating process: "The pilot’s 
commentary on the observations he made, and the 
rating he assigned. — They are the most important 
data on the closed-loop pilot-airplane combination 
which the engineer has." (Reference 1). The rating 
numbers themselves are an aeronautical short hand 
developed for recording, quantifying and analyzing 
subjective data. These ratings are a means to an end. 
They are not the end of the process. 

Engineering Test Pilots 

The scale in Figure 1 was developed for use by 
experimental and engineering test pilots. These test 
pilots typically have an operational background and 
have been trained to communicate with the engi- 
neering community. The military pilot becomes a test 
pilot after acquiring a personal understanding of the 
environment, threat and related friendly weapons 
systems which will define the total combat envi- 
ronment. They then learn (civil or military) to evaluate 
flying qualities in context with the cockpit workload 
with a readiness to deal with the environment and the 
adversity introduced by equipment failures. 

Pilot Comments 

Engineering test pilots are expected to know how to 
provide task ratings and comments which are useful in 
the analysis of the flights they conduct. It is not 
enough to provide a rating. The pilot must provide 
comments as to what the pilot experienced. The pilot 
must report what did and (sometimes) what did not 
influence the assignment of a given rating. For 
example, one pilot may use one technique to 
compensate for a lateral directional oscillation and be 
very successful, while a second pilot may not 
understand the best compensatory technique, have a 
great deal more trouble and assign a poor rating. 

Operational Pilots 

There are three probable situations where the 
operational pilots (unschooled in the methods of the 
engineering test pilots) could be expected to utilize the 
CHPRS. — In the ground based and inflight 
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ADEQUACY FOR SELECTED 
TASK OR REQUIRED OPERATION 


AIRCRAFT 

CHARACTERISTICS 


PILOT 

RATING 


DEMANDS ON THE 
PILOT IN SELECTED 
TASK OR REQUIRED 
OPERATION * 




Excellent 
Highly desirable 

Pilot compensation not a factor for 
desired performance. 



Good Negligible 

Pilot compensation not a factor for 

k L 

w 

deficiencies 

desired performance. 

YES 


Fair - Some midly 

unpleasant 

deficiencies 

Minimal pilot compensation 
required for desired performance. 


NO ^ ( Deficiencies 
warrant 
^improvement 



Minor but annoying 
deficiencies 

Desired performance requires 
moderate pilot compensation. 

Moderately objection- 
able deficiencies 

Adequate performance requires 
considerable pilot compensation. 

Very objectionable but 
tolerable deficiencies 

Adequate performance requires 
extensive pilot compensation. 


Major deficiencies 

Adequate performance not 
attainable with maximum tolerable 
pilot compensation. Controllability 
not in question. 

Major deficiencies 

Considerable pilot compensation 
ia required for control. 

Major deficiencies 

Intense pilot compensation Is 
required to retain control. 


Major deficiencies 


Control will be lost during some 
portion of required operation. 


* Definition of required operation involves designation of flight phase 
and sub-phases with accompanying conditions. 


Figure 1: The Cooper-Harper Pilot Rating Scale 


simulations cases, the resident simulation staff is very 
familiar with the use of the CHPRS and they are 
inclined to attempt to have the operational pilot use the 
CHPRS. The results of this application are potentially 
flawed because the operational pilots may not 
understand the proper use of the scale. — The scale 
looks simple, and these otherwise very capable pilots 
understate their lack of comprehension in an effort to 
be accommodating. 

In the operational evaluation venue, the resident 
engineers and analysts are much less familiar with the 
CHPRS and often hesitate to employ it Here an 
opportunity for broader use is missed 


In brief, the CHPRS is not sufficiently user friendly 
for many operational pilot applications unless the 
pilots and engineers are diligently trained in its use. 

The Scale 

The scale presented in Figure 1 incorporates 10 
ratings. Cooper and Harper feel that these ratings 
should be adequate for most evaluations (Reference 4). 
While they also recognize that the use of half rating 
gradation is appropriate for some applications (e.g. 3.5 
and 4.5 ), they discourage the practice. One reason for 
this reluctance is obvious. There are no definitions of 
half ratings. 
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Another argument against the use of half ratings 
asserts that ability of pilots to discriminate between 
flying qualities (workload and performance) is not 
sufficient to empower them to assign half ratings. The 
data in Figure 2 argues against this last assertion, for it 
contains a family of boundaries which separate areas 
of the flight envelope which were judged by an 
engineering test pilot to contain flying qualities that 
differ by one half of one PR. Boundaries of this sort 
were first identified in Reference 5, and later defined 
in flight with a small, modem helicopter. Over 60 
pilot ratings were recorded during stabilized, standard 
rate turning flight, while observing error limits of ± 5 
knots and ± 50 ft/m in. The actual ratings assigned to 
each area of the flight envelope vary as a function of 
the accompanying conditions (e.g., turbulence, 
lighting, visibility, etc.). 

Pilot Compensation/Workload Factors 

The level of pilot compensation necessary to achieve 
"adequate" or "desired" performance (see Figure 1) is 
integral to the use of CHPRS. Implicitly, this com- 
pensation is directly translatable to workload. 
Furthermore, the phrase "definition of required 
operation" (included in the asterisk note of the 
CHPRS) serves to include both direct flight control 
and other flight management functions which the pilot 
must perform to achieve satisfactory task performance. 


In the real world, the pilot approaches a flight task 
with the expectation that the task is doable. That is, 
pilots look at all of the sources of workload and 
attempt to cope with each source in the way which 
produces the best performance with a minimum of 
effort As Harper and Cooper observe in Reference 1, 
"the pilot adapts”. From the view of the systems 
engineer, the pilot learns how to achieve the desired 
performance while optimally distributing the piloting 
(handling qualities) workload and cockpit management 
(CM) workload. In military combat aircraft, mission 
equipment monitoring and task execution workload is 
also involved. 

The engineer understands that tasks are distributed by 
the crew in a natural attempt to avoid spikes in work- 
load which are likely to be accompanied by an 
unwanted dip in performance. It is this effective search 
for adaptive techniques which exemplifies the pilot's 
contribution to crew-machine performance. 

As the total workload builds, the pilot may have 
reason to periodically (albeit very briefly) allocate a 
high priority to CM tasks and allow errors in the flight 
path to build during a period of deferred attention. The 
performance during such unattended periods is 
therefore judged differently. The pilot who is prepared 
to allow an aircraft to drift off speed, or roll away from 
level flight, has substituted new (temporary) limits cm 



40 50 60 70 80 90 100 110 120 130 140 

INDICATED AIRSPEED (Knots) 

Figure 2: Boundaries of Flying Qualities Which Represent A Change of 0.5 Pilot Rating 
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the allowable flight path errors. These larger allowable 
errors apply only during the performance of the 
priority CM task. Typically, the pilot monitors the 
aircrafts departure from trim, and if every thing goes 
well, the CM task is completed during one period of 
unattended flight If the aircraft departs too quickly, 
or is difficult to return to trim, several periods of 
unattended or deferred flight control activity may be 
utilized to complete the CM task. Pilots evaluate such 
shared attention requirements and make a determi- 
nation as to suitability. 

Pilots also develop CM techniques which minimize the 
time required to accomplish CM tasks. For example, 
they learn how to identify switches by location, shape 
and mode of operation. This allows them to find a 
switch while focusing their eyes on a flight control 
task. The mind is obviously able to share its attention 
more rapidly than the eyes, especially when head 
movement is required. 

In addition, pilots who are faced with the need to use 
the right hand to conduct a CM task may use the left 
hand to control pitch and roll during the CM event. A 
pilot may also use a knee to hold a collective in 
position, or use both feet on the directional controls to 
keep the aircraft level in roll. Such techniques may 
result in substantially less deviation from the desired 
flight path with little or no increase in total workload. 
This is the way pilots learn to get the job done in the 
real world. Test pilots know these techniques and 
engineers need to report which ones they use. 

When pilots encounter a task which is not doable, 
many will attribute the failure to a personal inability. 
But, the more experienced the pilot, the less likely this 
will occur. Never- the- less, this is one more reason 
why it is very important for the analyst to understand 
the attitudes of subject pilots. 

In the vein of doable tasking, the "unexpected" 
typically places the ultimate stress on crew 
performance. The occurrence of unplanned events 
such as equipment malfunctions, unexpected route 
changes and unforecast weather are all a part of the 
equation. A totally correct evaluation of these events 
typically requires a concomitant engineering analysis 
to determine the probability of a given event. 

Defining The Task 

The CHPRS (Figure 1) contains a note which is often 
given less than adequate consideration. The note refers 
to the "task" or "operation" and alerts us to the effect: 
"Definition of required operation involves designation 
of flight phases and sub-phases with accompanying 
conditions." 

Flight Phases and Sub-Phases. If we translate the 
definitions of flight phase and sub phases as stated in 
Reference 4, we find that hovering flight and cruise 
flight are two typical flight phases. Activities 


associated with achieving a 40 ft hover is a sub phase. 
Maintaining a steady 40 ft hover is also a sub phase. 

Accompanying Conditions. The factors which 
collectively define "accompanying conditions" 
substantially influence the assignment and analysis of 
pilot ratings. Typically, the project engineer must 
define accompanying conditions prior to the flight for 
they at least partially define the test objective or 
"scope of test". The pilot needs this guidance to 
accomplish the desired evaluation. The actual accom- 
panying conditions, observed during the execution 
phase, must be recorded to support the best possible 
analysis and avoid unexplainable variance in the data. 

The factors which define some rotorcraft tasks can 
normally be selected from a list like the partial one 
presented below: 

(1) VMC or IMC task 

- type of cue field and display augmentation 

- display system 

(2) Performance Objectives 

- altitude (absolute or as measured by 

radar altimeter) 

- horizontal position enor (XandY) 

- heading variation limits 

- main transmission torque limits 

- engine operating limits 

- attitude variation limits during 

corrections ( ± degrees ) 

- attitude variation allowed as the result of 

a gust or turbulence 

- time available to conduct non flight 

control cockpit tasks (schedule of shared time) 

(3) Environmental Factors 

- underlying surface 

- near field visual screen 

- far field visual screen 

- near hazards-obstructions to hover 

- lighting 

- visual range 

- obstructions to visibility 

- precipitation 

- smote, fog. dust, snow, sun. 

- glare, sun, moon, reflections 

While most of the flying qualities community clearly 
understands the importance of the items listed under 
(1) and (2) above, the environmental factors under (3) 
seem to be less appreciated and are more often than 
not treated in too general a way. For example, limit 
environmental conditions are sometimes established 
by as few as one or two parameters (e.g., visibility). 
Such an abbreviated treatment is often inadequate, 
especially in the case of helicopters required to operate 
to and from a variety of fixed and moving platforms, 
in a rapidly changing air mass, day and night Figure 3 
was adapted from Reference 5 to expand on the list 
above and to illustrate the variety of conditions which 
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may be of interest during rotorcraft evaluations. While 
this figure is admittedly incomplete, figures like this 
should be provided so that pilots and engineers can 
accurately define sets of conditions for evaluation. In 
the real world, we find that rotorcraft pilots are 
interested in a variety of environmental conditions, 
any or all of which can represent a limit condition. 


Before leaving this subject, it is important to recognize 
that the introduction of "usable cue environments" in 
Reference 6 is an important contribution and a 
significant step in the right direction, as is the Navy's 
deck interface (DI) testing methodology which 
recognizes ship motion, lighting, wind, and other 
factors identified in Figure 3. 



Figure 3: Characteristics Defining Operational Environment 
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Figure 4: Probability Guidelines and Minimum HQ Requirements 


Probability of Encounter 

The probability of encountering adverse environmental 
factors is another important consideration when eval- 
uating the suitability of flying qualities and workload 
of a real aircraft It would appear that the probability 
of encountering certain environments can be treated in 
a way that is similar to the treatment of failure modes 
as addressed in References 6, 7, 8 and 9. 

In this regard, McElroy does an excellent job in 
Reference 10 of addressing and the probability of 
simultaneously encountering specific levels of 
atmospheric disturbance and failure states in context 
with flight envelopes. Figure 4 has been reproduced 
from Reference 10 as it is an excellent summary of the 
author's concept In support of this figure, the author 
observes that the FAA could use subjective pilot 
ratings (from a scale like that in Figure 1) to determine 
compliance with the criteria "satisfactory," "adequate," 
and "controllable" (Figure 4), an idea which is still 
new to much of the FAA. 


Analyzing Environmental Effects 
Plotting pilot radng data as a function of one or more 
variables will often help the analyst develop the 
highest degree of confidence in the data. This concept 
is demonstrated in Reference 11 which presents a 
family of six data plots (one each for 5, 10, 15, 20, 25, 
30 knots of wind), two of which are characterized in 
Figure 5. Observe that pilot ratings are plotted as a 
function of azimuth for two wind speeds. 

Note that pilot ratings vary as a function of both wind 
speed and azimuth. Although not shown, the ratings 
can also vary as a function of gross weight, power 
available, center of gravity, rotor RPM, turbulence, 
visibility, lighting, and a host of other variables. If you 
inspect the 5 and 15 knot data for the wind azimuth of 
300°, you will note that the pilot rating changes from a 
respectable PR 3 at 5 knots of wind to a relatively poor 
rating of PR 5 at 15 knots. But why? The pilots 
comments should provide the best insight. This is a 
clear demonstration of the need for pilot comments. 



180° 

5KTAS RELATIVE WIND 


_ 150° 

180° 

15 KTAS RELATIVE WIND 


Figure 5: Handling Qualities For Various Wind Azimuth Angles (Pre-landing hover) Over Deck of Small Ships 
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Evaluating Simulation Facilities 
Pilot ratings can also be used to evaluate the authen- 
ticity of a simulator. One way to check the authentici- 
ty of the simulation is to ask the crews to evaluate (or 
interpret) the simulator visual and motion systems 
while simulating an aircraft with which they are 
familiar. 

To illustrate this application , the results of a 
hypothetical simulator evaluation are presented here as 
Figure 6. This figure contains the possible result of a 
day flight and a night flight in an existing helicopter 
followed by an attempt to replicate the real world test 
conditions in a ground based simulator. The data 
shown for "Bright Day - Actual Flight" in this figure is 
taken directly from Figure 5. In this illustration, the 
pilot's "actual flight" PRs and "simulated flight" PRs 
are approximately equal for the dark night case, but 
the data for the bright day case reveals a significant 
disagreement The comments accompanying the pilot's 
ratings should confirm the ratings and provide insight 
into the probable cause. Depending upon the 
comparative evaluation of the pilot's control activity 
and overall performance, the findings would seem to 
suggest that the visual representation lacked adequate 
authenticity in the "bright day" case. In contrast, the 
dim, night scene was adequate. This is an important 
finding in and of itself. 

The results of a second hypothetical evaluation are 
presented in Figure 7 which illustrates an alternative 
format for evaluating the authenticity of a simulator. 
In this case, the pilot first uses a real helicopter to 
conduct a demanding task in seven different, real 
world environments. The seven combinations have 
been plotted in ascending order for convenience. 

When the same pilot attempts the identical task in the 
simulated environments (duplicated in the simulator), 
the pilot ratings should agree. If they do not agree, the 
pilots comments associated with each rating should 
provide useful data as to the cause of the difficulty. 


That is, an analyses of pilot control activity, attitude 
error, flight path error, etc., should include an equally 
exhaustive analysis of pilot comments. 

Minimizing Variability In Ratings 

Variance in PR data feeds the argument that the 
subjective rating approach can produce erroneous 
results. Cooper-Harper tell us to expect a limited 
amount of variability in ratings. Disparity in pilot 
background can produce variation in the pilot ratings. 

In addition, Cooper-Harper tell us that some pilots 
may be predisposed for or against a given configu- 
ration. In addition, some variability in PRs may simply I 

reflect the presence of one or more factors) which ! 

were not accounted for in the definition of the j 

experiment. That is, an important factor may not j 

have been recorded. I 

Most of these sources of variability can be minimized 
through diligent planning. In particular, pilots and 
engineers are urged develop a table such as that 
included as Figure 3. Once the data are collected, 
presentations formats such as those suggested by I 

Figures 5, 6 and 7 can help the analyst develop the j 

best possible understanding of PR data and at the same § 

time minimize the possibility of scatter in the data. 1 

Extrapolations j 

The CHPRS authors recognize that some would have I 

pilots evaluate only the situation experienced (first 
hand) by the subject pilot. Others would have pilots 
use a simulator evaluation experience to predict/- - 

extrapolate to the real world. For example, assume that l 

during a landing experiment, employing an inflight 
simulator, the pilot evaluates the test configuration 1 

only on a clear, bright sunny day. The pilot could then | 

be asked to rate only the situation flown, (clear day to j 

a runway), or alternately, the pilot could be asked to 1 

extrapolate the clear day observations into a dark, wet 
night environment 



Figure 6: Comparison Of Pilot Ratings To Evaluate A Simulation Facility 
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Figure 7: An Example Set of Progressively More Difficult Environmental Conditions Which Can be Evaluated 
in the Real World and Replicated in a Simulator to Collect Pilot Ratings for Evaluation of a Simulation Facility 


Cooper-Harper agree that a pilot can extrapolate this 
experience and provide a rating for an environment 
worse than that observed in a hands-on evaluation. 
This of course assumes that the pilot has acquired an 
adequate understanding of the aircraft and is familiar 
with the operational environment of interest Cooper- 
Harper go on to ask the question "... if the pilot doesn’t 
do it, who will?". They also go on to conclude that an 
experienced pilot is probably the best qualified to 
extrapolate simulator experience into the real world. 

The same ability to extrapolate has been recognized 
and utilized in the military and FAA evaluations of 
aircraft for at least forty years. That is, an experienced 
pilot is often asked to conclude in a few flights, that a 
given aircraft is, or is not, suitable for flight into 
instrument conditions without ever flying into 
instrument conditions. Regardless of the approach 
taken, the pilot and engineer should agree on which 
approach they will use and this selection should be 
reported with the data. This note of caution is 
supported by Harper and Cooper in Reference 1 . 

WORKLOAD AND INTEGRATED 
EVALUATIONS 

With the increased use of computer based systems, the 
pilot's task has shifted more and more towards the 
overall flight management function. In minimum crew 


(one or two place) military combat aircraft, these 
system advances have added to the mission system 
functions over which the pilot has direct control. For 
military aircraft, the same technology advances have 
greatly expanded the functions of non-pilot air crew 
mission system operators - and increased the thrust 
towards the use of a minimum crew. 

On the civil side, there is the potential of single pilot 
IFR helicopter operations, including approaches to 
busy airports and slow speed steep approaches into 
confined landing sites. These operations bring a 
similar concern for increased cockpit complexity and 
higher workload. 

Workload 

These developments have increased the attention of 
specialists in human task performance to the measure- 
ment of pilot and air crew workload, with recognition 
that in-flight measurement is needed to fully 
characterize the actual experience. Issues of objective 
vs. subjective measurement have received continuing 
attention in this field as elaborated in References 12 
and 13. Due to the complexity of the total in-flight 
workload and the intrusiveness of available objective 
measurement techniques, there has been increasing 
acceptance of and support for subjective 
measurement. 
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DIFFICULTY LEVEL OPERATOR DEMAND LEVEL RATING 



Figure 8: Modified Cooper-Harper Scale 
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OPERATOR DECISIONS 









Subjective measurements schemes that have been 
evaluated (Reference 14) indicate that some of them, 
while useful in laboratory investigations of pilot or 
operator workload, are quite cumbersome and too time 
consuming in ground based flight simulator or in-flight 
use. Chambers and Hilmer in Reference 14 clearly 
show the advantages of Weirwille's proposed Modified 
Cooper-Harper Rating Scale (MCHRS) (Figure 8) for 
workload assessment in both piloting and non-piloting 
tasks in these applications. The brief treatment in that 
paper, however, does not go on to point out other 
benefits of its use for these applications. 

For example, the familiarity of engineering test pilots, 
simulation staff and flight test engineering personnel 
with the CHPRS provides a direct carry over to use of 
the MCHRS. This should allow its use in both 
workload measurement per se and in the evaluation of 
non-piloting flight management and mission systems, 
either individually or as part of the integrated overall 
pilot/pilot and mission specialist task. In the case of 
these applications, strictures similar to those which 
accompany the CHPRS would have to be developed. 
This would including the need for subject comments 
similar to those provided by pilots in the CHPRS. 

Need For Single Integrated Rating 
While separate assessments of handling qualities and 
workload can be useful in research investigations, for 
example Reference 15 contains the results of one such 
effort, this approach fails to give the decision maker a 
readily usable answer regarding operational suitability. 

Decision makers need an overall rating which reflects 
the total suitability of the aircraft to accomplish its 
mission when operated by the typical air crew for 
which the aircraft was designed. For civil aircraft, 
FAA certification is the final go/no go decision. For 
military aircraft, the formal Operational Evaluation is 
the final stamp. But the use of the CHPRS to 
primarily evaluate flying qualities (with consideration 
of flying and CM workload inferred) and the use of the 
MCHRS as a sub-set to the CHPRS to evaluate 
workload, does not provide the desired single rating. 
It also fails to deal with comparative priorities (e.g., 
the flying task vs. the CM task). Another approach is 
needed. 

INTERPRETED COOPER-HARPER PILOT 
RATING SCALE 

Introduction 

The preceding discussion has suggested that there is a 
need to apply the CHPRS more broadly while 
observing the strictures more diligently. This includes 
the need for a scale which is easier for operational 
pilots to use and which treats workload a bit more 
directly. This need includes both flying and the non- 
flying, cockpit management workload and the related 
priorities. In addition, there is the need to define half 
pilot ratings. 


An example of how all of this might be accomplished 
is presented in Figure 9. The Interpreted Cooper- 
Harper Pilot Rating Scale (ICHPRS), as addressed 
here, is meant to have the same meaning as the 
original CHPRS of References 1 and 4. The concept 
also applies to the entire scale, but a complete 
treatment is beyond the scope of this paper. When 
compared to the CHPRS in Figure 1, it is quickly 
obvious that the first "pilot decision" steps are not 
included in Figure 9. In military version of this scale, 
these pilot decisions steps would be retained. In the 
civil version, they might not be retained (as suggested 
in Reference 16). 

Half PRs 

As discussed earlier, half PRs accomplish two 
objectives. First, they allow pilots to evaluate a condi- 
tion or situation which does not meet the definition of 
a whole number in the CHPRS. Second, the half 
ratings allow the pilot or analyst to build a higher 
degree of confidence as to where the boundaries of 
interest are located. But the CHPRS does not provide 
definitions and, depending upon the application, this 
can represent a serious problem. 

In contrast, the ICHPRS does include definitions for 
half ratings. These half ratings relate to the preceding 
whole integer rating and not to the subsequent rating. 
The logic of this approach is more apparent when one 
considers the transition between PRs of 3 and 4, and 
the PRs of 6 and 7 (especially when considering many 
of the military applications). Civil evaluators may 
draw the lines of suitability elsewhere with the same 
concern. 

"Use Unique" Interpretative Narrative 
The narrative in Figure 9 is meant to suggest an 
approach, not "the only" or "the recommended" 
approach. In most cases, the narrative in the ICHPRS 
should be developed by one or more engineering test 
pilot(s) and engineers) familiar with the test aircraft, 
its operational characteristics, and its operational 
requirements. This should produce one or more 
aircraft-mission unique scale(s), depending upon the 
scope of the evaluation. 

The added descriptors might evolve during the initial 
shake down of an aircraft or during a familiarization 
period in the aircraft or in a simulator (if a real aircraft 
is not available). In any event, the use of a trained 
engineering test pilot, familiar with the Cooper-Harper 
scale, is strongly recommended. 

Note that the descriptions under "Aircraft 
Characteristics" in Figure 9 are identical to those 
found in the CHPRS presented in Figure 1. The 
narrative under "To Achieve the best attainable 
performance” has two parts. The first part (left 
column) repeats the descriptions found in the CHPRS. 
The second part (right column) is split into two 
horizontal boxes, TTiese two boxes contain the inter- 
pretive narrative for one whole PR and the associated 
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half pilot rating. Note that this second column contains 
comments relating to both flight control and CM task- 
ing, including indications of priority and performance. 

The final column (under Representative Observations) 
amplifies the preceding descriptions of the pilot effort 
required by characterizing performance in terms of 
operational suitability. Here, examples are provided to 
aid the pilot in efforts to discriminate. As described in 
References 1 and 4, failure to meet the intent of any 
specific rating forces the pilot to assign the next higher 
rating. 

Performance. Performance objectives must be de- 
fined prior to commencing an evaluation. These objec- 
tives must relate to tasks for which the pilot expects to 
achieve minimum error, or for situations where the 
pilot desires to maximize time out of the loop to rest or 
to conduct a CM task during which some amount of 
flight path error is acceptable. For example, this might 
characterize the shared monitoring of the aircraft's 
flight path and a mission equipment display. 

In this regard, CM tasking could to be evaluated to 
determine the critical tasks and the procedures which 
apply. What CM tasks must be accomplished during 
high gain flight control events? This includes 
consideration of failure modes. For example, if an 
engine fails, is the pilot expected to continue the task 


and deal with the emergency procedures, or does the 
pilot first transition to a new flight phase? 

The interpretative narrative can include detailed refer- 
ences to performance expectations or objectives for 
both the flying and the CM tasks. That is, there is no 
reason why performance objectives should not be in- 
serted in the narrative. It seems likely that this 
approach would reduce the potential for variance, but 
in some situations, this level of detail would probably 
not be necessary. 

Definitions. Once the performance objectives have 
been defined, and the narrative has been drafted, 
definitions should be developed and supported with 
examples where required. 

Performance Priority. The narrative in Figure 9 was 
developed with the idea that the flight path 
performance of the aircraft was the primary or critical 
objective. It is also possible to have situations where 
CM is of paramount concent. The narrative would be 
written appropriately for such flight phases to reflect 
these changing priorities. For example, it may be 
important accomplish an electronic warfare task in a 
very precise and timely way, while operating at 
altitudes and speeds which minimize concern for flight 
path error. 


AIRCRAFT 

CHARACTERISTICS 

DEMANDS ON THE PILOT IN SELECTED 
TASK OR REQUIRED OPERATION * 

PILOT ** 
RATING 



To Achlava tha boot attainable performance. 

Representative Observations 


Minor, But 
Annoying 
Characteristics 

uesirea 
performance 
requires moderate 
pilot compensation. 

i Pilot most concentrate on flight path 
j errors. CM tasks are accomplished 
| following standard procedures. 

Occasional relaxed control is possible, but workload 
sometimes results in unwanted deviation. Pilot is 
impatient and fatigued during extended operations. 
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j Pilot must concentrate on flight path 
! errors. CM procedures are altered to 
! accommodate reducted pilot capacity 
[ to monitor cockpit status. 

Relaxed control is unachievable. Considerable 
compensation sometimes required. Pilot quickly 
impatient, quickly fatigued. Not accepted as the 
norm for the duration of routine or probable flight. 
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Adequate performance 
requires considerable 
pilot compensation. 

Pilot attention is fully focused on early 
error detection. Pilot is often unable 
to effectively plan and execute cockpit 
management tasking in accordance 
with standard procedures. 

Performance is marginal for a precision task 
and is not acceptable for routine or probable 
operations. Pilot does not have the time to 
adequately monitor status of CM tasking. 
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: Concentration on error detection and 
compensation is intense, and 
j approaching limit Many cockpit 
! management tasks are deferred, some 
are precluded, 
i 

Maximum acceptable compensation is required. 
Unusual attitude may develop while 
accomplishing CM task. Pilot is confident of 
success during 15 min precision and 120 min of 
improbable operations pursuing a non-precision. 

5.5 

Very 

Objectionable 
But Tolerable 
Characteristics 

Adequate performance 
requires extensive pilot 
compensation. 

; Concentration on flying task is at 
j limit Critical CM activities are 
j accomplished randomly, as 
! opportunities arise during 
; momentary improvement in flying 
; task performance. 

Excessive pilot compensation is required to 
continue marginally safe operations for 5 min 
in precision task and 30 to 60 min in non-precision 
tasks. Pilot is occasionally alarmed at combinations 
of error, error buildup rate and total workload. 
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j Concentration on flying task is at 
j limit Adequate flight performance 
■ can not be attained if any CM tasking 
• is undertaken. 

t 

t 

i 

i 
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Compensation is at limit. Acceptable performance 
will probably only be achieved during very brief 
periods ranging from seconds to a minute. Pilot will 
persist only if there is no safer alternative. Aircraft 
will probably not be damaged is pilot persists. If 
pilot attempts Of task, aircraft may incur minor 
damage. 

6.5 


* Definition of required operation involves designation of flight phase and sub-phases with accompanying conditions. 
** If a mission-flight critical cockpit management task can not be accomplished in a timely and effective way, the PR = 7. 


Figure 9: An Example of Interpretive Narrative Added to a Portion of the Cooper-Harper Pilot Rating Scale 
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Figure 10: NVG Pilot Rating Analysis Classifications 


NIGHT VISION TESTING BY FAA 
This paper was in part made possible as the result of 
work funded by the FAA Rotorcraft Research Program 
Office in Washington, D.C., and accomplished in 
support of flight evaluations of night vision devices 
conducted by the Flight Test Division of the FAA 
Technical Center, Atlantic City International Airport, 
New Jersey. The objective of the evaluation was to 
provide an opportunity for a large number of civil and 
FAA pilots to fly with night vision goggles (NVGs) to 
determine their suitability for use by EMS operators. 

This evaluation was chartered to use a group of civil 
helicopter pilots with dissimilar flying backgrounds to 
examine the safety of flight issues associated with the 
use of NVGs while operating in a variety of 
environments. For example, a variety of lighting 
environments and obstructions to visibility were of 
interest. — None of the evaluation tasks involved Nap 
of the Earth (NOE) flying techniques. 

As a result, a set of evaluation guides (booklets) were 
developed to help introduce pilots to the evaluation, 
and to help them understand an early ICHPRS. 
(References 17, 18, and 19). The interpreted pilot 
rating scale was meant to be faithful to the intent of the 
CHPRS. This project is currendy underway and the 
results are yet to be documented. — The current plan 
is to sort the pilot rating assessment data and compile 
the results for each task in a way which is 


characterized above in Figure 10. This should provide 
decision makers with data they need to determine 
suitability in terms of pilot experience and 
environmental factors. 

CONCLUSIONS AND OBSERVATIONS 
Cooper-Harper is an effective subjective assessment 
tool when applied in accordance with its creators full 
instructions. Extensive successful use in the past, and 
the evolving "test and approval decision processes" are 
areas where its effectiveness can be enhanced for 
current and future applications. 

The ability of pilots to extrapolate pilot ratings is a 
well proven capability which is essential to safe, 
affordable and timely evaluation of aircraft and 
simulations of proposed aircraft designs. 

A suitably tailored Interpreted Cooper-Harper Rating 
Scale as proposed will provide pilots not having an 
engineering test pilot background with an effective 
bating system for use in simulations and final 
operational evaluations. 

The effectiveness of using Cooper-Harper in handling 
qualities evaluations, where workload is a factor in the 
assessment, strongly supports the use of a proposed 
Modified Cooper-Harper, appropriately adapted, in 
specific subjective workload assessment and non-pilot 
airborne system evaluations. 
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